Skip to content

feat(functions): add unicodeRegExp option to schema core function#2809

Open
mtso wants to merge 5 commits into
stoplightio:developfrom
mtso:add-unicoderegexp-option
Open

feat(functions): add unicodeRegExp option to schema core function#2809
mtso wants to merge 5 commits into
stoplightio:developfrom
mtso:add-unicoderegexp-option

Conversation

@mtso
Copy link
Copy Markdown

@mtso mtso commented Apr 17, 2025

Fixes #2419.

Checklist

  • Tests added / updated
  • Docs added / updated

Does this PR introduce a breaking change?

  • Yes
  • No

Screenshots

Before:

Screenshot 2025-04-17 at 8 09 46 PM

After:

Screenshot 2025-04-17 at 8 09 02 PM

Added unicodeRegExp to core function docs
Screenshot 2025-04-17 at 8 21 44 PM

Additional context

Adds an option unicodeRegExp to the schema core function. Defaults to "false". When "true", uses the unicode flag "u" with "pattern" and "patternProperties" fields during schema validation; otherwise does not use the "u" flag when "false" or omitted.

The unicodeRegExp option is optional, and thus backwards compatible with the existing behavior of spectral's schema core function. When the option is omitted, the behavior is defaulted to "false".

The unicodeRegExp option is also added as an optional flag to oasSchema and oasExample oas functions. To preserve the existing behavior of the standard oas ruleset, unicodeRegExp: false has been explicitly added to the standard oas ruleset's duplicated-entry-in-enum, oas2-valid-schema-example, oas2-valid-media-example, oas3-valid-media-example and oas3-valid-schema-example rules.

Adding this option allows rulesets that use the schema core function to opt into using the unicode "u" flag when validating the "pattern" and "patternProperty" fields.

Adding this option to oasExample and oasSchema also allows rulesets that extend from spectral:oas to replace the oas3-valid-media-example and oas3-valid-schema-example rules with a version that opts into using unicode regular expressions. Doing so is currently a little involved, though, requiring re-specifying the given block of the rules as shown below. But it at least makes the option available to extending rulesets.

// functions/oasExample.js
import { default as oasExample } from '@stoplight/spectral-rulesets/dist/oas/functions/oasExample';

export default oasExample;
# spectral.yaml
extends: ['spectral:oas']

functions:
- 'oasExample'

rules:
  oas3-valid-schema-example:
    given:
      - $.components.schemas..[?(@property !== 'properties' && @ && (@ && @.example !== void 0 || @.default !== void 0) && (@.enum || @.type || @.format || @.$ref || @.properties || @.items))]
      - $..content..[?(@property !== 'properties' && @ && (@ && @.example !== void 0 || @.default !== void 0) && (@.enum || @.type || @.format || @.$ref || @.properties || @.items))]
      - $..headers..[?(@property !== 'properties' && @ && (@ && @.example !== void 0 || @.default !== void 0) && (@.enum || @.type || @.format || @.$ref || @.properties || @.items))]
      - $..parameters..[?(@property !== 'properties' && @ && (@ && @.example !== void 0 || @.default !== void 0) && (@.enum || @.type || @.format || @.$ref || @.properties || @.items))]
    then:
      function: oasExample
      functionOptions:
        schemaField: '$'
        oasVersion: 3
        type: 'schema'
        unicodeRegExp: true

  oas3-valid-media-example:
    given:
      - '$..content..[?(@ && @.schema && (@.example !== void 0 || @.examples))]'
      - '$..headers..[?(@ && @.schema && (@.example !== void 0 || @.examples))]'
      - '$..parameters..[?(@ && @.schema && (@.example !== void 0 || @.examples))]'
    then:
      function: oasExample
      functionOptions:
        schemaField: 'schema'
        oasVersion: 3
        type: 'media'
        unicodeRegExp: true

@mtso mtso requested a review from a team as a code owner April 17, 2025 12:44
@mtso mtso changed the title Add unicodeRegExp option to schema core function feat(functions): Add unicodeRegExp option to schema core function Apr 17, 2025
@mtso mtso changed the title feat(functions): Add unicodeRegExp option to schema core function feat(functions): add unicodeRegExp option to schema core function Apr 17, 2025
@mtso
Copy link
Copy Markdown
Author

mtso commented Apr 24, 2025

@mnaumanali94 / @frankkilcommins , when you get the chance, could you please take a look?

@mtso mtso force-pushed the add-unicoderegexp-option branch from d02c697 to a66cd15 Compare September 19, 2025 13:25
@jwulf
Copy link
Copy Markdown

jwulf commented Mar 12, 2026

@mnaumanali94 @frankkilcommins @mtso — any chance we can get this merged?

Copy link
Copy Markdown
Contributor

@slegarraga slegarraga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Useful addition for schemas with Unicode property names — unicodeRegExp opt-in on the schema function keeps default behavior unchanged (#2419).

Ajv instance caching keyed by options is a nice touch; tests + docs updated. LGTM.

pull Bot pushed a commit to nagyist/camunda-zeebe that referenced this pull request Jun 3, 2026
## Description

Switch the `openapi-lint` CI job from upstream `@stoplight/spectral-cli`
+ pinned `@stoplight/spectral-rulesets` to the Camunda fork
[`@camunda8/spectral-cli`](https://www.npmjs.com/package/@camunda8/spectral-cli).

The fork is a single pre-bundled package (no transitive dependency on
`@stoplight/spectral-rulesets`) that fixes three upstream issues:

1. **nimma null-deref** — the `duplicated-entry-in-enum` rule crashes on
`null` example values. The fork adds null guards to the JSONPath filter.
2. **Unicode regex false positives** — Ajv rejects valid Unicode
patterns (e.g. `\p{L}`) because `unicodeRegExp` defaults to `false`. The
fork integrates [PR
camunda#2809](stoplightio/spectral#2809) and defaults
`unicodeRegExp` to `true`.
3. **`ignoreUnknownFormat` at ruleset level** — allows rulesets to
suppress `unrecognized-format` warnings without requiring CLI flags.

Fork repo: https://github.com/camunda/camunda-spectral

## Changes

### CI workflow (`.github/workflows/ci.yml`)
- Replace `@stoplight/spectral-cli@6.16.0` +
`@stoplight/spectral-rulesets@1.22.2` with
`@camunda8/spectral-cli@6.16.1`
- Simplify install step to a single `npm install -g` call
- Remove the `spectral-rulesets` pinning workaround comment

### Spectral ruleset (`.spectral.yaml`)
- Add `ignoreUnknownFormat: true` to suppress `unrecognized-format`
warnings on YAML partials (eliminates 43 warnings on the file-level
pass)
- Promote `oas3-valid-media-example` from `warn` to `error` (all media
example issues are now fixed)
- Keep `oas3-valid-schema-example` at `warn` — ScopeKey uses `oneOf`
with structurally identical branches (`ProcessInstanceKey`,
`ElementInstanceKey`), causing "must match exactly one schema in oneOf"
on any ScopeKey example. Changing `oneOf` to `anyOf` would fix the
warning but breaks Java code generation. 4 warnings remain from this.
- Remove the `TODO` comment about restoring severity (no longer needed
for media examples)

### OpenAPI spec fixes
- `decision-definitions.yaml`: Fix example `decisionDefinitionId` from
`1234-5678` (starts with digit, violates `^[\p{L}_]...` pattern) to
`my-decision`
- `system.yaml`: Add `type: string` to nullable `stage` property
(Spectral requires `type` alongside `nullable`), change example from
`null` to `"prod"`

### What was NOT changed (and why)
- `keys.yaml` ScopeKey `oneOf` — left as-is because changing to `anyOf`
alters the generated Java classes and breaks the build (confirmed by CI
failure on earlier push)
- `process-instances.yaml` `ancestorElementInstanceKey` `oneOf` — same
reason; changing to `allOf` breaks codegen

Closes camunda#54071
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Spectal complains when a openapi pattern uses a Unicode character set

3 participants